<html>
<head><meta charset="utf-8"><title>Build times and build systems · general · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/index.html">general</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html">Build times and build systems</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="217944835"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217944835" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217944835">(Nov 25 2020 at 22:06)</a>:</h4>
<p>So, introductions of a sort: I'm the dev lead at Microsoft for a team whose sole focus is enabling Rust development in Windows and Azure.  Compile times are one of our biggest blockers for major adoption.  Which is why I'm so interested in parallel builds right now.</p>



<a name="217944892"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217944892" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217944892">(Nov 25 2020 at 22:07)</a>:</h4>
<p>In addition to being a Cargo team member and language team co-lead, I'm now the founder of a company making compile times faster for everyone. Nice to meet you. :)</p>



<a name="217944966"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217944966" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217944966">(Nov 25 2020 at 22:08)</a>:</h4>
<p>Nice!  We'll probably have some design work in Cargo to share with you, in the coming months. Integrating stuff into the Windows build system has...  unique challenges.</p>



<a name="217944983"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217944983" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217944983">(Nov 25 2020 at 22:08)</a>:</h4>
<p>I'm looking forward to hearing about that.</p>



<a name="217944999"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217944999" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217944999">(Nov 25 2020 at 22:08)</a>:</h4>
<p>What <em>is</em> the primary Windows build system these days?</p>



<a name="217945007"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217945007" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217945007">(Nov 25 2020 at 22:08)</a>:</h4>
<p>Er, one moment first.</p>



<a name="217945094"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217945094" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Notification Bot <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217945094">(Nov 25 2020 at 22:09)</a>:</h4>
<p>This topic was moved here from <a class="stream-topic" data-stream-id="131828" href="/#narrow/stream/131828-t-compiler/topic/Making.20Symbol.20more.20efficient.3A.20PR.20feedback.3F">#t-compiler &gt; Making Symbol more efficient: PR feedback?</a> by <span class="user-mention silent" data-user-id="239881">Josh Triplett</span></p>



<a name="217945150"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217945150" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217945150">(Nov 25 2020 at 22:10)</a>:</h4>
<p>OK, I split this branch of the discussion to avoid derailing your PR discussion thread.</p>



<a name="217945239"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217945239" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217945239">(Nov 25 2020 at 22:10)</a>:</h4>
<p>We just call it <code>build.exe</code>. It's the same <code>build.exe</code> that we publish with the Windows SDK / DDK (back when there was a DDK).  Maybe we don't publish it any more.  It is...  not a good build system.  I've actually designed a better build system, which was part of an internal project, and which morphed into what we called Domino. It's public name is BuildXL.</p>



<a name="217945325"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217945325" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217945325">(Nov 25 2020 at 22:11)</a>:</h4>
<p><code>build.exe</code> is mainly a scheduling front-end for <code>nmake.exe</code>. Again, these are not systems that I would recommend to _anyone_ these days, but we're stuck with it for a variety of reasons.</p>



<a name="217945343"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217945343" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217945343">(Nov 25 2020 at 22:11)</a>:</h4>
<p>I've seen buildxl.</p>



<a name="217945456"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217945456" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217945456">(Nov 25 2020 at 22:12)</a>:</h4>
<p>I can think of a few potential issues integrating Cargo into build.exe / nmake. What are you running into?</p>



<a name="217945488"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217945488" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217945488">(Nov 25 2020 at 22:12)</a>:</h4>
<p>Our main constraints involving Rust / Cargo are around supply chain security (binary provenance, not reaching out to the Internet for crate source, etc.).  That, and how to schedule Cargo in the middle of a much larger build schedule.</p>



<a name="217945516"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217945516" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217945516">(Nov 25 2020 at 22:13)</a>:</h4>
<p>We have an internal mirror of <a href="http://crates.io">crates.io</a>, which solves some problems, but not all.</p>



<a name="217945643"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217945643" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217945643">(Nov 25 2020 at 22:14)</a>:</h4>
<p>Taking those one at a time...</p>



<a name="217945674"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217945674" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217945674">(Nov 25 2020 at 22:15)</a>:</h4>
<p>The other problems are in how to identify crates in our ecosystem, not just in <a href="http://crates.io">crates.io</a>. The Windows codebase is spread among a number of Git repos, these days.  Most of it is in one massive mega-repo (the largest Git repo in existence, I think), plus a bunch of other repos.  So we need ways to say "I need to refer to crate blar_blar from internal repo $REPO with credentials $C", and things like that.  Some of that maps to existing Cargo concepts, some doesn't.</p>



<a name="217945698"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217945698" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217945698">(Nov 25 2020 at 22:15)</a>:</h4>
<p>If you use a monorepo, could you vendor all the crates into your repository, and then use <code>--offline --frozen</code>?</p>



<a name="217945733"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217945733" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217945733">(Nov 25 2020 at 22:15)</a>:</h4>
<p>(racing, there; interesting that it isn't <em>completely</em> a monorepo...)</p>



<a name="217945818"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217945818" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217945818">(Nov 25 2020 at 22:16)</a>:</h4>
<p>Credentials <em>can</em> be handled out-of-band so you don't have to check them into the repository, if that helps.</p>



<a name="217945839"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217945839" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217945839">(Nov 25 2020 at 22:16)</a>:</h4>
<p>Or you can refer to things by path, and/or use submodules.</p>



<a name="217945966"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217945966" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217945966">(Nov 25 2020 at 22:18)</a>:</h4>
<p>Right. Some things are _mostly_ working the way we need.  I don't have a concrete list of "we need Cargo to change in these ways" yet, but I'd really like to work with you &amp; the community on how to meet these needs. We're making fairly serious commitments to Rust, internally, and I really want the full Rust experience: Cargo, <a href="http://crates.io">crates.io</a>, etc. and not just "well, we kinda have it working".</p>



<a name="217945982"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217945982" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217945982">(Nov 25 2020 at 22:18)</a>:</h4>
<p>Absolutely, and I <em>really</em> appreciate that your first reaction is "let's improve Cargo" rather than "let's replace Cargo".</p>



<a name="217946005"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946005" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946005">(Nov 25 2020 at 22:18)</a>:</h4>
<p>That would potentially make you the largest company that <em>doesn't</em> have their custom build system almost completely bypass Cargo. :)</p>



<a name="217946042"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946042" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946042">(Nov 25 2020 at 22:19)</a>:</h4>
<p>Vendoring sources into our monorepo is similar to how we used to do things with C/C++, but we want to go in the opposite direction. We've been pulling C/C++ OSS sources _out_ of the Windows tree, and into separate repositories, for lots of good reasons.  Mainly de-dup'ing, license management, automatic upgrades or at least audits of new releases of code from the wild.</p>



<a name="217946141"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946141" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946141">(Nov 25 2020 at 22:20)</a>:</h4>
<p>That makes sense. So, you want to have a collection of adjacent repositories, and use crates from them, without having to vendor them.</p>



<a name="217946166"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946166" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946166">(Nov 25 2020 at 22:21)</a>:</h4>
<p>Our mono-repo is an artifact of Windows history. We used to run the largest cluster of P4-derived repos in the world. Then we moved that giant mess (I MEAN mass) to Git. We've been slowly "undocking" components by moving them into separate repos, and then wiring them together with CI pipelines.</p>



<a name="217946183"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946183" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946183">(Nov 25 2020 at 22:21)</a>:</h4>
<blockquote>
<p>So, you want to have a collection of adjacent repositories</p>
</blockquote>
<p>Precisely.</p>



<a name="217946197"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946197" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946197">(Nov 25 2020 at 22:21)</a>:</h4>
<p>Submodules might help you, and might also substantially complicate your life.</p>



<a name="217946271"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946271" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946271">(Nov 25 2020 at 22:22)</a>:</h4>
<p>Ideally our internal ecosystem would look a lot like <a href="http://crates.io">crates.io</a>. We want the experience for developers who come to Microsoft to be "oh, this is just normal Rust, I feel productive immediately".</p>



<a name="217946284"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946284" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946284">(Nov 25 2020 at 22:22)</a>:</h4>
<p>I think I might be unfamiliar with what you mean by submodules. Can you expand on that some?</p>



<a name="217946379"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946379" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946379">(Nov 25 2020 at 22:24)</a>:</h4>
<p><span class="user-mention silent" data-user-id="274471">Arlie Davis</span> <a href="#narrow/stream/122651-general/topic/Build.20times.20and.20build.20systems/near/217946271">said</a>:</p>
<blockquote>
<p>Ideally our internal ecosystem would look a lot like <a href="http://crates.io">crates.io</a>. We want the experience for developers who come to Microsoft to be "oh, this is just normal Rust, I feel productive immediately".</p>
</blockquote>
<p>Then I think you did the right thing by <em>starting</em> with a private <a href="http://crates.io">crates.io</a> mirror, and you may also want your own separate registry where you can upload custom internal crates (keep it separate from the mirror). We could definitely do better, but we could also do a <em>lot</em> worse, so I think that's a great start.</p>



<a name="217946442"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946442" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946442">(Nov 25 2020 at 22:24)</a>:</h4>
<p><span class="user-mention silent" data-user-id="274471">Arlie Davis</span> <a href="#narrow/stream/122651-general/topic/Build.20times.20and.20build.20systems/near/217946284">said</a>:</p>
<blockquote>
<p>I think I might be unfamiliar with what you mean by submodules. Can you expand on that some?</p>
</blockquote>
<p>git submodules. Git supports adding, to a repository, an entry for "this directory should contain commit hash abc123". And then you can supply metadata saying "get that from <code>https://.../foo/bar.git</code> if you don't have it".</p>



<a name="217946466"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946466" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946466">(Nov 25 2020 at 22:25)</a>:</h4>
<p>Oh, Git submodules. I thought there was some Cargo notion of submodules.</p>



<a name="217946475"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946475" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946475">(Nov 25 2020 at 22:25)</a>:</h4>
<p>Ah. No, none.</p>



<a name="217946629"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946629" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946629">(Nov 25 2020 at 22:27)</a>:</h4>
<p>If you're splitting up a monorepo into separate repositories, git submodules would be one way to handle it. The upside is that <em>semantically</em> it's probably very close to what you want. The downside is that it substantially increases the amount of git expertise required to work on the repository successfully, plus you'd have to make sure your custom layers atop git work with it.</p>



<a name="217946640"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946640" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eh2406 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946640">(Nov 25 2020 at 22:27)</a>:</h4>
<p>Somewhat off topic but I think <span class="user-mention" data-user-id="120054">@Jon Gjengset</span> is working on similar things at amazon. So you should compare notes. If Microsoft and Amazon say "adding fet x to Cargo will make are hacks less ugly" that is a big push.</p>



<a name="217946712"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946712" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946712">(Nov 25 2020 at 22:28)</a>:</h4>
<p>I've had mixed experiences with submodules. For $REASONS, they're also just not an option for our mono-repo. We'd also much rather manage semver references in Cargo.toml files than submodules.</p>



<a name="217946722"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946722" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946722">(Nov 25 2020 at 22:28)</a>:</h4>
<p>I think "mixed" describes almost everyone's experience with submodules. ;)</p>



<a name="217946727"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946727" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946727">(Nov 25 2020 at 22:28)</a>:</h4>
<p>Heh!</p>



<a name="217946732"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946732" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946732">(Nov 25 2020 at 22:28)</a>:</h4>
<p>And yeah, semver sounds better.</p>



<a name="217946773"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946773" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946773">(Nov 25 2020 at 22:29)</a>:</h4>
<p>So, it might be that a custom Cargo registry plus some git-based CI and links to repositories would help.</p>



<a name="217946781"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946781" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946781">(Nov 25 2020 at 22:29)</a>:</h4>
<p>Also, we're going to be stuck with our mono-repo for a very long time. We're slowly pulling components out of it, but whenever we do, we always find a long tail of dependencies. So it's a slow process, and a lot of things will probably never be pulled out. My wider team (the Windows Engineering Systems team) handles a lot of that.</p>



<a name="217946795"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946795" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946795">(Nov 25 2020 at 22:29)</a>:</h4>
<blockquote>
<p>So, it might be that a custom Cargo registry plus some git-based CI and links to repositories would help.</p>
</blockquote>
<p>Exactly.</p>



<a name="217946866"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946866" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946866">(Nov 25 2020 at 22:30)</a>:</h4>
<p>One question we've had for a while: What's the thinking of the Cargo team on "namespaces" for <a href="http://crates.io">crates.io</a>?  Namespaces tied to organizations, provenance, etc.</p>



<a name="217946876"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946876" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946876">(Nov 25 2020 at 22:30)</a>:</h4>
<p>We're actually talking about that right now.</p>



<a name="217946911"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946911" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946911">(Nov 25 2020 at 22:31)</a>:</h4>
<p>There's a proposal on the table for very simple namespacing: if you own the crate <code>foo</code>, you can have crates named <code>foo/bar</code> and nobody else can.</p>



<a name="217946936"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946936" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946936">(Nov 25 2020 at 22:31)</a>:</h4>
<p><span class="user-mention" data-user-id="132040">@Manish Goregaokar</span> is working on that.</p>



<a name="217946948"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946948" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Arlie Davis <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946948">(Nov 25 2020 at 22:31)</a>:</h4>
<p>Seems like a pure win.</p>



<a name="217946996"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217946996" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217946996">(Nov 25 2020 at 22:32)</a>:</h4>
<p>The Cargo team's general thinking is that the set of problems it addresses need some thought, and we don't set <a href="http://crates.io">crates.io</a> policy directly, but we're happy to provide the features in Cargo that enable new <a href="http://crates.io">crates.io</a> policies.</p>



<a name="217947003"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217947003" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217947003">(Nov 25 2020 at 22:32)</a>:</h4>
<p><span class="user-mention silent" data-user-id="274471">Arlie Davis</span> <a href="#narrow/stream/122651-general/topic/Build.20times.20and.20build.20systems/near/217946948">said</a>:</p>
<blockquote>
<p>Seems like a pure win.</p>
</blockquote>
<p>I agree.</p>



<a name="217961641"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Build%20times%20and%20build%20systems/near/217961641" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jon Gjengset <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Build.20times.20and.20build.20systems.html#217961641">(Nov 26 2020 at 03:09)</a>:</h4>
<p>Just to follow on from this, yes, I'll be fiddling a lot with Rust and Cargo integration at Amazon in the coming months, and like you <span class="user-mention" data-user-id="274471">@Arlie Davis</span> my goals is for "Rust at Amazon" to be ~= "Rust everywhere" by working _with_ upstream, so I think it's likely we'll be pushing in similar directions. Yay! Your description of having a local mirror and trying to make that fit into your pipeline sounds like it fits some of the stuff I've been looking at too :) See also <a href="https://internals.rust-lang.org/t/alternate-registry-package-rebuilds/13409">https://internals.rust-lang.org/t/alternate-registry-package-rebuilds/13409</a> and <a href="https://github.com/rust-lang/cargo/pull/8890">https://github.com/rust-lang/cargo/pull/8890</a> for some relevant bits and pieces.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>